Position and Gesture Touch Sensor Responsive to Displacements and Angles of At Least One Contacting Finger

ABSTRACT

A touch-based user interface system comprising an arrangement for scanning a tactile sensor array to produce a corresponding array of measurement values that are presented to a processor for computation. The computation produces a plurality of running sums created from selected measurement values or functions of selected measurement values. A post-scan computation algorithm derives at least three independently-adjustable interactive control parameters responsive to at least displacements or angles the contact of a single area of threshold contact or threshold proximity. The system provides output control signals responsive to the independently-adjustable interactive control parameters. In one aspect of the invention, n algorithmic element for handling of regions of threshold contact or threshold proximity having non-convex shapes. In another aspect of the invention, an algorithmic element calculates the rate of change of one or more of the independently-adjustable interactive control parameters. Other aspects of the invention include shape and gesture recognition.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.11/761,978, filed Jun. 12, 2007, which is a continuation of U.S.application Ser. No. 09/812,400, filed Mar. 19, 2001, now U.S. Pat. No.7,786,370, issued Aug. 31, 2010, which is a division of U.S. applicationSer. No. 09/313,533, filed May 15, 1999, now U.S. Pat. No. 6,610,917,issued Aug. 26, 2003, which claims benefit of priority of U.S.provisional application Ser. No. 60/085,713, filed May 15, 1998.

This application is also related to U.S. application Ser. No.13/470,725, filed May 14, 2012.

FIELD OF INVENTION

The present invention relates generally to a control system, and inparticular, to a tactile input controller for controlling an associatedsystem.

SUMMARY OF THE INVENTION

Touchpad user interfaces for controlling external systems such ascomputers, machinery, and process environments via at least threeindependent control signals. The touchpad may be operated by hand, otherparts of the body, or inanimate objects. Such an interface affords awide range of uses in computer applications, machine and processcontrol, and assistance to the disabled. In one embodiment simplecontact position-sensing touchpads, producing control signals responsiveto a contact region, are enhanced to provide several independent controlsignals. Enhancements may include velocity sensors, pressure sensors,and electronic configurations measuring contact region widths.Touch-screens positioned over visual displays may be adapted. Accordingto other aspects pressure-sensor array touchpads are combined with imageprocessing to responsively calculate parameters from contact regions.Six independent control parameters can be derived from each region ofcontact. These may be easily manipulated by a user. In oneimplementation, smaller pressure-sensor arrays are combined with dataacquisition and processing into a chip that can be tiled in an array.

DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the presentinvention will become more apparent upon consideration of the followingdescription of preferred embodiments taken in conjunction with theaccompanying drawing figures, wherein:

FIG. 1 shows an example of how two independent contact points can beindependently discerned, or the dimensional-width of a single contactpoint can be discerned, for a resistance null/contact controller with asingle conductive contact plate or wire and one or more resistiveelements whose resistance per unit length is a fixed constant througheach resistive element;

FIG. 2 shows how a pressure-sensor array touch-pad can be combined withimage processing to assign parameterized interpretations to measuredpressure gradients and output those parameters as control signals;

FIG. 3 illustrates the positioning and networking of pressure sensingand processing “mini-array” chips in larger contiguous structures;

FIG. 4 illustrates the pressure profiles for a number of example handcontacts with a pressure-sensor array;

FIG. 5 illustrates how six degrees of freedom can be recovered from thecontact of a single finger; and

FIG. 6 illustrates examples of single, double, and quadruple touch-padinstruments with pads of various sizes and supplemental instrumentelements.

DETAILED DESCRIPTION Overview

Described herein are two kinds of novel touch-pads. Null/contacttouchpads are contact-position sensing devices that normally are in anull state unless touched and produce a control signal when touchedwhose signal value corresponds to typically one unique position on thetouch-pad. A first enhancement is the addition of velocity and/orpressure sensing. A second enhancement is the ability to either discerneach dimensional-width of a single contact area or, alternatively,independently discern two independent contact points in certain types ofnull/contact controllers. A third possible enhancement is that ofemploying a touch-screen instance of null/contact touch pad andpositioning it over a video display.

The invention also provides for a pressure-sensor array touch-pad. Apressure-sensor array touch-pad of appropriate sensitivity range,appropriate “pixel” resolution, and appropriate physical size is capableof measuring pressure gradients of many parts of the human hand or footsimultaneously. A pressure-sensor array touch-pad can be combined withimage processing to assign parameterized interpretations to measuredpressure gradients and output those parameters as control signals. Thepressure-sensor “pixels” of a pressure-sensor array are interfaced to adata acquisition stage; the data acquisition state looks for sensorpixel pressure measurement values that exceed a low-levelnoise-rejection/deformity-reject threshold; contiguous regions ofsufficiently high pressure values are defined; the full collection ofregion boundaries are subjected to classification tests; variousparameters are derived from each independent region; and theseparameters are assigned to the role of specific control signals whichare then output to a signal routing, processing, and synthesis entity.

It is possible to derive a very large number of independent controlparameters which are easily manipulated by the operating user. Forexample, six degrees of freedom can be recovered from the contact of asingle finger. A whole hand posture can yield 17 instantaneously andsimultaneously measurable parameters which are independently adjustableper hand. The recognized existence and/or derived parameters frompostures and gestures may be assigned to specific outgoing controlsignal formats and ranges. The hand is used throughout as an example,but it is understood that the foot or even other body regions, animalregions, objects, or physical phenomena can replace the role of thehand.

It will be evident to one of ordinary skill in the art that it isadvantageous to have large numbers of instantaneously and simultaneouslymeasurable parameters which are independently adjustable. For instance,a symbol in a 2-D CAD drawing can be richly interactively selected andinstalled or edited in moments as opposed to tens to hundreds of secondsas is required by mouse manipulation of parameters one or two at a timeand the necessary mode-changes needed to change the mouse actioninterpretation. As a result, said touch-pad has applications in computerworkstation control, general real-time machine control, computer dataentry, and computer simulation environments.

Various hardware implementations are possible. A particularlyadvantageous implementation would be to implement a smallpressure-sensor array together with data acquisition and a smallprocessor into a single chip package that can be laid as tiles in alarger array.

Null/Contact Touch-Pads

Distinguished from panel controls and sensors are what will be termednull/contact touch-pads. This is a class of contact-position sensingdevices that normally are in a null state unless touched and produce acontrol signal when touched whose signal value corresponds to typicallyone unique position on the touch-pad. Internal position sensingmechanisms may be resistive, capacitive, optical, standing wave, etc.Examples of these devices include one-dimensional-sensing ribboncontrollers found on early music synthesizers, two-dimensional-sensingpads such as the early Kawala pad and more modern mini-pads found onsome lap-top computers, and two-dimensional-sensing see-throughtouch-screens often employed in public computer kiosks.

The null condition, when the pad is untouched, requires and/or providesthe opportunity for special handling. Some example ways to handle theuntouched condition include:

-   -   sample-hold (hold values issued last time sensor was touched, as        does a joystick)    -   bias (issue maximal-range value, minimal-range value, mid-range        value, or other value)    -   touch-detect on another channel (i.e., a separate out-of-band        “gate” channel).

Additional enhancements can be added to the adaptation of null/contacttouch-pad controllers as instrument elements. A first enhancement is,the addition of velocity and/or pressure sensing. This can be done viaglobal impact and/or pressure-sensors. An extreme of this isimplementation of the null/contact touch-pad controller as apressure-sensor array; this special case and its many possibilities aredescribed later.

A second enhancement is the ability to either discern eachdimensional-width of a single contact area or alternatively,independently discern two independent contact points in certain types ofnull/contact controllers. FIG. 1 shows an example of how two independentcontact points can be independently discerned, or the dimensional-widthof a single contact point can be discerned, for a resistancenull/contact controller with a single conductive contact plate (as withthe Kawala pad product) or wire (as in a some types of ribbon controllerproducts) and one or more resistive elements whose resistance per unitlength is a fixed constant through each resistive element. It isunderstood that a one-dimensional null/contact touch-pad typically hasone such resistive element while a two-dimensional null/contacttouch-pad typically has two such resistive elements that operateindependently in each direction.

Referring to FIG. 1, a constant current source can be applied to theresistive element as a whole, developing a fixed voltage across theentire resistive element. When any portion of the resistive element iscontacted by either a non-trivial contiguous width and/or multiplepoints of contact, part of the resistive element is shorted out, thusreducing the overall width-to-end resistance of the resistance element.Because of the constant current source, the voltage developed across theentire resistive element drops by an amount equal to the portion of theresistance that is shorted out.

The value of the voltage drop then equals a value in proportion to thedistance separating the extremes of the wide and/or multiple contactpoints. By subtracting the actual voltage across the entire resistiveelement from the value this voltage is normally, a control voltageproportional to distance separating the extremes of the wide and/ormultiple contact points is generated. Simultaneously, the voltagedifference between that of the contact plate/wire and that of the end ofthe resistive clement closest to an external contact point is stillproportional to the distance from said end to said external contactpoint. Using at most simple op-amp summing and/or differentialamplifiers, a number of potential control voltages can be derived; forexample one or more of these continuously-valued signals:

-   -   value of distance difference between external contact points (or        “width”; as described above via constant current source, nominal        reference voltage, and differential amplifier)    -   center of a non-trivial-width region (obtained by simple        averaging, i.e., sum with gain of ½)    -   value of distance difference between one end of the resistive        element and the closest external contact point (simple        differential amplifier)    -   value of distance difference between the other end of the        resistive element and the other external contact point (sum        above voltage with “width” voltage with appropriate sign).

Further, through use of simple threshold comparators, specificthresholds of shorted resistive element can be deemed to be, forexample, any of a single point contact, a recognized contact regionwidth, two points of contact, etc., producing correspondingdiscrete-valued control signals. The detection of a width can be treatedas a contact event for a second parameter analogous to the singlecontact detection event described at the beginning. Some example usagesof these various continuous and discrete signals are:

-   -   existence of widths or multiple contact points may be used to        trigger events or timbre changes    -   degree of widths may be used to control degrees of modulation or        timbre changes    -   independent measurement of each extremal contact point from the        same end of the resistive element can be used to independently        control two parameters. In the simplest form, one parameter is        always larger than another; in more complex implementations, the        trajectories of each contact point can be tracked (using a        differentiator and controlled parameter assignment switch); as        long as they never simultaneously touch, either parameter can        vary and be larger or smaller than the other.

It is understood that analogous approaches may be applied to othernull/contact touchpad technologies such as capacitive or optical.

A third possible enhancement is that of employing a touch-screeninstance of null/contact touch-pad and positioning it over a videodisplay. The video display could for example provide dynamicallyassigned labels, abstract spatial cues, spatial gradients, line-of-sitecues for fixed or motor-controlled lighting, etc. which would bevaluable for use in conjunction with the adapted null/contact touch-padcontroller.

These various methods of adapted null/contact touch-pad elements can beused stand-alone or arranged in arrays. In addition, they can be used asa component or addendum to instruments featuring other types ofinstrument elements.

Pressure-Sensor Array Touch-Pads

The invention provides for use of a pressure-sensor array arranged as atouch-pad together with associated image processing. As with thenull/contact controller, these pressure-sensor array touch-pads may beused stand-alone or organized into an array of such pads.

It is noted that the inventor's original vision of the below describedpressure-sensor array touch-pad was for applications not only in musicbut also for computer data entry, computer simulation environments, andreal-time machine control, applications to which the below describedpressure-sensor array touch-pad clearly can also apply.

A pressure-sensor array touch-pad of appropriate sensitivity range,appropriate “pixel” resolution, and appropriate physical size is capableof measuring pressure gradients of many parts of the flexibly-rich humanhand or foot simultaneously. FIG. 2 shows how a pressure sensor arraytouch-pad can be combined with image processing to assign parameterizedinterpretations to measured pressure gradients and output thoseparameters as control signals.

The pressure-sensor “pixels” of a pressure-sensor array touch-pad 1300are interfaced to a data acquisition stage 1301. The interfacing methodmay be fully parallel but in practice may be advantageously scanned at asufficiently high rate to give good dynamic response to rapidly changinghuman touch gestures. To avoid the need for a buffer amplifier for eachpressure-sensor pixel, electrical design may carefully balance parasiticcapacitance of the scanned array with the electrical characteristics ofthe sensors and the scan rates; electrical scanning frequencies can bereduced by partitioning the entire array into distinct parts that arescanned in parallel so as to increase the tolerance for address settlingtimes and other limiting processes.

Alternatively, the pressure-sensor array 1300 may be fabricated in sucha way that buffer amplifier arrays can be inexpensively attached to thesensor array 1300, or the sensors may be such that each contains its ownbuffer amplifier; under these conditions, design restrictions onscanning can be relaxed and operate at higher speeds. Although thepressure sensors may be likely analog in nature, a further enhancementwould be to use digital-output pressure-sensor elements or sub-arrays.

The data acquisition stage 1301 looks for sensor pixel pressuremeasurement values that exceed a low-levelnoise-rejection/deformity-rejection threshold. The sufficiently highpressure value of each such sensor pixel is noted along with therelative physical location of that pixel (known via the pixel address).This noted information may be stored “raw” for later processing and/ormay be subjected to simple boundary tests and then folded intoappropriate running calculations as will be described below. In general,the pressure values and addresses of sufficiently high pressure valuepixels are presented to a sequence of processing functions which may beperformed on the noted information:

-   -   contiguous regions of sufficiently high pressure values are        defined (a number of simple run-time adjacency tests can be        used; many are known—see for example [Ronse; Viberg; Shaperio;        Hara])    -   the full collection of region boundaries are subjected to        classification tests; in cases a given contiguous region may be        split into a plurality of tangent or co-bordered independently        recognized regions    -   various parameters are derived from each independent region, for        example geometric center, center of pressure, average pressure,        total size, angle-of-rotation-from reference for non-round        regions, second-order and higher-order geometric moments,        second-order and higher-order pressure moments, etc.    -   assignment of these parameters to the role of specific control        signals (note events, control parameters, etc.) which are then        output to a signal routing, processing, and synthesis entity;        for example, this may be done in the form of MIDI messages.

Because of the number of processes involved in such a pipeline, it isadvantageous to follow a data acquisition stage 1301 with one or moreadditional processing stages 1303, 1305, 1309, and 1311. Of the fourexample processing functions just listed, the first three fall in thecharacter of image processing. It is also possible to do a considerableamount of the image processing steps actually within the dataacquisition step, namely any of simple adjacency tests and foldingselected address and pressure measurement information into running sumsor other running pre-calculations later used to derive aforementionedparameters. The latter method can be greatly advantageous as it cansignificantly collapses the amount of data to be stored.

Regardless of whether portions of the image processing are done withinor beyond the data acquisition stage, there are various hardwareimplementations possible. One hardware approach would involve verysimple front-end scanned data acquisition hardware and a singlehigh-throughput microprocessor/signal-processor chip. Alternatively, anexpanded data acquisition stage may be implemented in high-performancededicated function hardware and this would be connected to a lowerperformance processor chip. A third, particularly advantageousimplementation would be to implement a small pressure-sensor arraytogether with data acquisition and a small processor into a singlelow-profile chip package that can be laid as tiles in a nearly seamlesslarger array. In such an implementation all image processing could infact be done via straightforward partitions into message-passingdistributed algorithms.

One or more individual chips could direct output parameter streams to anoutput processor which would organize and/or assign parameters to outputcontrol channels, perhaps in a programmable' manner under selectablestored program control. A tiled macro array of such “sensor mini-array”chips could be networkedsby a tapped passive bus, one- ortwo-dimensional mode active bus daisy-chain, a potentially expandablestar-wired centralized message passing chip or subsystem, or othermeans.

Creating a large surface from such “tile chips” will aid in theserviceability of the surface. Since these chips can be used as tiles tobuild a variety of shapes, it is therefore possible to leverage asignificant manufacturing economy-of-scale so as to minimize cost andjustify more extensive feature development. Advanced seating andconnector technologies, as used in laptops and other high-performanceminiature consumer electronics, can be used to minimize the separationbetween adjacent chip “tiles” and resultant irregularities in thetiled-surface smoothness. A tiled implementation may also include a thinrugged flexible protective film that separates the sensor chips from theoutside world. FIG. 3 illustrates the positioning and networking ofpressure sensing and processing “mini-array” chips 1400 in largercontiguous structures 1410.

With the perfection of a translucent pressure-sensor array, it furtherbecomes possible for translucent pressure-sensor arrays to be laid atopaligned visual displays such as LCDs, florescent, plasma, CRTs, etc. aswas discussed above for null/contact touch-pads. The displays can beused to label areas of the sensor array, illustrate gradients, etc. Notethat in the “tile chip” implementation, monochrome or color displayareas may indeed be built into each chip.

Returning now to the concept of a pressure-sensor array touch-pad largeenough for hand-operation: examples of hand contact that may berecognized, example methods for how these may be translated into controlparameters, and examples of how these all may be used are now described.In the below the hand is used throughout as an example, but it isunderstood that the foot or even other body regions, animal regions,objects, or physical phenomena can replace the role of the hand in theseillustrative examples.

FIG. 4 illustrates the pressure profiles for a number of example handcontacts with a pressure-sensor array. In the case 1500 of a finger'send, pressure on the touch-pad pressure-sensor array can be limited tothe finger tip, resulting in a spatial pressure distribution profile1501; this shape does not change much as a function of pressure.Alternatively, the finger can contact the pad with its flat region,resulting in light pressure profiles 1502 which are smaller in size thanheavier pressure profiles 1503. In the case 1504 where the entire fingertouches the pad, a three-segment pattern (1504 a, 1504 b, 1504 c) willresult under many conditions; under light pressure a two segment pattern(1504 b or 1504 c missing) could result. In all but the lightestpressures the thumb makes a somewhat discernible shape 1505 as do thewrist 1506, cuff 1507, and palm 1508; at light pressures these patternsthin and can also break into disconnected regions. Whole hand patternssuch as the first 1511 and flat hand 1512 have more complex shapes. Inthe case of the first 1511, a degree of curl can be discerned from therelative geometry and separation of sub-regions (here depicted, as anexample, as 1511 a, 1511 b, and 1511 c). In the case of the whole flathand 1512, there can be two or more sub-regions which may be in factjoined (as within 1512 a) and/or disconnected (as an example, as 1512 aand 1512 b are); the whole hand also affords individual measurement ofseparation “angles” among the digits and thumb (1513 a, 1513 b, 1513 c,1513 d) which can easily be varied by the user.

Relatively simple pattern recognition software can be used to discernthese and other hand contact patterns which will be termed “postures.”The pattern recognition working together with simple image processingmay, further, derive a very large number of independent controlparameters which are easily manipulated by the operating user. In manycases it may be advantageous to train a system to the particulars of aspecific person's hand(s) and/or specific postures. In other situationsthe system may be designed to be fully adaptive and adjust to a person'shand automatically. In practice, for the widest range of control andaccuracy, both training and ongoing adaptation may be useful. Further,the recognized postures described thus far may be combined in sequencewith specific dynamic variations among them (such as a finger flick,double-tap, etc.) and as such may be also recognized and thus treated asan additional type of recognized pattern; such sequential dynamics amongpostures will be termed “gestures.”

The admission of gestures further allows for the derivation ofadditional patterns such as the degree or rate of variation within oneor more of the gesture dynamics. Finally, the recognized existenceand/or derived parameters from postures and gestures may be assigned tospecific outgoing control signal formats and ranges. Any traininginformation and/or control signal assignment information may be storedand recalled for one or more players via stored program control.

For each recognized pattern, the amount of information that can bederived as parameters is in general very high. For the human hand orfoot, there are, typically, artifacts such as shape variation due toelastic tissue deformation that permit recovery of up to all six degreesof freedom allowed in an object's orientation in 3-space.

FIG. 5 illustrates how six degrees of freedom can be recovered from thecontact of a single finger. In the drawing, the finger 1600 makescontact with the touch-pad 1601 with its end segment at a point on thetouch-pad surface determined by coordinates 1611 and 1612 (these wouldbe, for example, left/right for 1611 and forward/backward for 1612).Fixing this point of contact, the finger 1600 is also capable ofrotational twisting along its length 1613 as well as rocking back andforth 1614. The entire finger can also be pivoted with motion 1615 aboutthe contact point defined by coordinates 1611 and 1612. These are allclearly independently controlled actions, and yet it is still possiblein any configuration of these thus far five degrees of freedom, to varythe overall pressure 1616 applied to the contact point. Simple practice,if it is even needed, allows the latter overall pressure 1616 to beindependently fixed or varied by the human operator as other parametersare adjusted.

In general other and more complex hand contacts, such as use of twofingers, the whole hand, etc. forfeit some of these example degrees offreedom but often introduce others. For example, in the quiteconstrained case of a whole hand posture, the fingers and thumb canexert pressure independently (5 parameters), the finger and thumbseparation angles can be varied (4 parameters), the finger ends 1504 acan exert pressure independently from the middle 1504 b and inner 1504 csegments (4 parameters), the palm can independently vary its appliedpressure (1 parameter) while independently tilting/rocking in twodirections (2 parameters) and the thumb can curl (1 parameter), yielding17 instantaneously and simultaneously measurable parameters which areindependently adjustable per hand. Complex contact postures may also beviewed as, or decomposed into, component sub-postures (for example here,as flat-finger contact, palm contact, and thumb contact) which wouldthen derive parameters from each posture independently. For such complexcontact postures, recognition as a larger compound posture which maythen be decomposed allows for the opportunity to decouple and/orrenormalize the parameter extraction in recognition of the specialaffairs associated with and constraints imposed by specific complexcontact postures.

It is noted that the derived parameters may be pre-processed forspecific uses. One example of this would be the quantization of aparameter into two or more discrete steps; these could for example besequentially interpreted as sequential notes of a scale or melody.Another example would be that of warping a parameter range as measuredto one with a more musically expressive layout.

Next examples of the rich metaphorical aspects of interacting with thepressuresensor array touch-pad are illustrated. In many cases there maybe one or more natural geometric metaphor(s) applicable, such asassociating left-right position, left-right twisting, or left-rightrotation with stereo panning, or in associating overall pressure withvolume or spectral complexity. In more abstract cases, there may bepairs of parameters that go together—here, for example with a fingerend, it may be natural to associate one parameter pair with (left/rightand forward/backward) contact position and another parameter pair with(left/right and forward/backward) twisting/rocking. In this latterexample there is available potential added structure in the metaphor byviewing the twisting/rocking plane as being superimposed over theposition plane. The superposition aspect of the metaphor can be viewedas an index, or as an input-plane/output-plane distinction for atwo-input/two-output transformation, or as two separated processes whichmay be caused to converge or morph according to additional overallpressure, or in conjunction with a dihedral angle of intersectionbetween two independent processes, etc.

Next, examples of the rich syntactical aspects of interacting with thepressure-sensor array touch-pad are illustrated. Some instruments haveparticular hand postures naturally associated with their playing. It isnatural then to recognize these classical hand-contact postures andderive control parameters that match and/or transcend how a classicalplayer would use these hand positions to evoke and control sound fromthe instrument. Further, some postures could be recognized either inisolation or in gestural-context as being ones associated with (orassigned to) percussion effects while remaining postures may beassociated with accompanying melodies or sound textures.

As an additional syntactic aspect, specific hand postures and/orgestures may be mapped to specific selected assignments of controlsignals in ways affiliated with specific purposes. For example, fingerends may be used for one collection of sound synthesis parameters, thumbfor a second potentially partially overlapping collection of soundsynthesis parameters, flat fingers for a third partially-overlappingcollection, wrist for a fourth, and cuff for a fifth, and first for asixth. In this case it may be natural to move the hand through certainconnected sequences of motions; for example: little finger end, still incontact, dropping to flat-finger contact, then dropping to either palmdirectly or first to cuff and then to palm, then moving to wrist, allnever breaking contact with the touch-pad. Such permissible sequences ofpostures that can be executed sequentially without breaking contact withthe touch-pad will be termed “continuous grammars.”

Under these circumstances it is useful to set up parameter assignments,and potentially associated context-sensitive parameter renormalizations,that work in the context of selected (or all available) continuousgrammars. For example, as the hand contact evolves as being recognizedas one posture and then another, parameters may be smoothly handed-overin interpretation from one posture to another without abrupt changes,while abandoned parameters either hold their last value or return to adefault value (instantly or via a controlled envelope).

Now a number of example applications of the pressure-sensor arraytouchpad are provided. It is known to be possible and valuable to usethe aforementioned pressure-sensor array touch-pad, implicitlycontaining its associated data acquisition, processing, and assignmentelements, for many, many applications such as general machine controland computer workstation control. One example of machine control is inrobotics: here a finger might be used to control a hazardous materialrobot hand as follows:

-   -   left/right position: left/right hand position    -   in/out position: in/out hand position    -   in/out rock: up/down hand position    -   rotation: hand grip approach angle    -   overall pressure: grip strength    -   left/right twist: gesture to lock or release current grip from        pressure control

A computer workstation example may involve a graphical Computer-AidedDesign application currently requiring intensive mouse manipulation ofparameters one or two at a time:

-   -   left/right position: left/right position of a selected symbol in        a 2-D CAD drawing    -   in/out position: up/down position of a selected symbol in 2-D        CAD drawing    -   left/right twist: symbol selection—left/right motion through 2-D        palette    -   in/out rock: symbol selection—up/down motion through 2-D palette    -   rotation: rotation of selected symbol in the drawing    -   overall pressure: sizing by steps    -   tap of additional finger: lock selection into drawing or unlock        for changes    -   tap of thumb: undo    -   palm: toggle between add new object and select existing object

Clearly a symbol can be richly interactively selected and installed oredited in moments as opposed to tens to hundreds of seconds as isrequired by mouse manipulation of parameters one or two at a time andthe necessary mode-changes needed to change the mouse actioninterpretation.

Touch-Pad Array

Touch-pad instrument elements, such as null/contact types andpressure-sensor array types described earlier, can be used in isolationor arrays to create electronic controller instruments. The touch-pad(s)may be advantageously supplemented with panel controls such as pushbuttons, sliders, knobs as well as impact sensors forvelocity-controlled triggering of percussion or pitched note events. Ifone or more of the touch-pads is transparent (as in the case of anull/contact touch screen overlay) one or more video, graphics, oralphanumeric displays 2711 may placed under a given pad or group ofpads.

FIG. 6 illustrates examples of single 2710, double 2720, and quadruple2730 touchpad instruments with pads of various sizes. A single touch-padcould serve as the central element of such an instrument, potentiallysupplemented with panel controls such as push buttons 2714, sliders2715, knobs 2716 as well as impact sensors. In FIG. 6, a transparent padsuperimposed over a video, graphics, or one or more alphanumericdisplays is assumed, and specifically shown is a case of underlaygraphics cues being displayed for the player. Two large sensors can beput side by side to serves as a general purpose left-hand/right-handmulti-parameter controller.

All publications and patent applications mentioned in this specificationare herein incorporated by reference to the same extent as if eachindividual publication or patent application was specifically andindividually indicated to be incorporated by reference. The inventionnow being fully described, it will be apparent to one of ordinary skillin the art that many changes and modifications can be made theretowithout departing from its spirit or scope.

REFERENCES CITED

The following references are cited in this patent application using theformat of the first one or two authors last name(s) within squarebrackets“[ ]”, multiple references within a pair of square bracketsseparated by semicolons “;”

[Ronse] Ronse, Christian and Devijver, Pierre A., Connected Componentsin Binary Images: the Detection Problem, John Wiley & Sons Inc. NewYork, 1984;

[Viberg] Viberg, Mats, Subspace Fitting Concepts in Sensor ArrayProcessing, Linkoping Studies in Science and Technology. DissertationsNo. 27 Linkoping, Sweden 1989;

[Shapiro] Shapiro, Larry S, Affine Analysis of Image Sequences,Cambridge University Press, 1995;

[Hara] Hara, Yoshiko “Matsushita demos multilayer MPEG-4 compression”,Electronic Engineering Times, Apr. 19, 1999.

1. A touch-based user interface system comprising: a tactile sensorarray; an arrangement for scanning the tactile sensor array, the scanproducing a corresponding array of measurement values; a processor forcalculating of a plurality of running sums from selected measurementvalues or functions of selected measurement values; a post-scancomputation algorithm for deriving at least threeindependently-adjustable interactive control parameters responsive to atleast displacements or angles the contact of a single area of thresholdcontact or threshold proximity; and an algorithmic element for handlingof regions of threshold contact or threshold proximity having non-convexshapes, wherein the system provides output control signals, the controlsignals responsive to the independently-adjustable interactive controlparameters.
 2. The system of claim 1 wherein the tactile sensor array isa pressure sensor array.
 3. The system of claim 1 wherein the singlearea of threshold contact or threshold proximity is that of a singlefinger tip.
 4. The system of claim 1 wherein the system furthercomprises: an algorithmic element for identifying a plurality ofisolated regions of threshold contact or threshold proximity, and analgorithmic element for each of the isolated region of threshold contactor threshold proximity the computation of a plurality of running sumsfrom selected measurement values or functions of selected measurementvalues for that respective isolated region of threshold contact orthreshold proximity, wherein the system derives a plurality ofindependently-adjustable interactive control parameters from eachisolated region of threshold contact or threshold proximity of aplurality of parts of the hand.
 5. The system of claim 1 wherein thesystem further comprises an algorithmic element for calculating the rateof change of one or more of the independently-adjustable interactivecontrol parameters.
 6. The system of claim 1 wherein the system furthercomprises an algorithmic element implementing shape recognitionfunctions.
 7. The system of claim 1 wherein the system further comprisesan algorithmic element implementing gesture recognition functions. 8.The system of claim 1 wherein the system further comprises analgorithmic element for calculating one or more symbols.
 9. The systemof claim 1 wherein the system further comprises an algorithmic elementimplementing syntax functions.
 10. The system of claim 1 wherein thetactile sensor array is comprises a plurality of partitioned sections.11. A touch-based user interface system comprising: a tactile sensorarray; an arrangement for scanning the tactile sensor array, the scanproducing a corresponding array of measurement values; a processor forcomputation of a plurality of running sums from selected measurementvalues or functions of selected measurement values; a post-scancomputation algorithm for deriving at least threeindependently-adjustable interactive control parameters responsive to atleast displacements or angles the contact of a single area of thresholdcontact or threshold proximity; and an algorithmic element forcalculating the rate of change of one or more of theindependently-adjustable interactive control parameters, wherein thesystem provides output control signals, the control signals responsiveto the independently-adjustable interactive control parameters.
 12. Thesystem of claim 11 wherein the tactile sensor array is a pressure sensorarray.
 13. The system of claim 11 wherein the single area of thresholdcontact or threshold proximity is that of a single finger tip.
 14. Thesystem of claim 11 wherein the system further comprises: an algorithmicclement for identifying a plurality of isolated regions of thresholdcontact or threshold proximity; and an algorithmic element for each ofthe isolated region of threshold contact or threshold proximity thecomputation of a plurality of running sums from selected measurementvalues or functions of selected measurement values for that respectiveisolated region of threshold contact or threshold proximity, wherein thesystem derives a plurality of independently-adjustable interactivecontrol parameters from each isolated region of threshold contact orthreshold proximity of a plurality of parts of the hand.
 15. The systemof claim 11 wherein the system further comprises an algorithmic elementfor handling of regions of threshold contact or threshold proximityhaving non-convex shapes.
 16. The system of claim 11 wherein the systemfurther comprises an algorithmic element implementing shape recognitionfunctions.
 17. The system of claim 11 wherein the system furthercomprises an algorithmic element implementing gesture recognitionfunctions.
 18. The system of claim 11 wherein the system furthercomprises an algorithmic element for calculating one or more symbols.19. The system of claim 11 wherein the system further comprises analgorithmic element implementing parsing functions.
 20. The system ofclaim 11 wherein the tactile sensor array is comprises a plurality ofpartitioned sections.